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Abstract. Mobile apps can access a wide variety of secnre information, 
such as contacts and location. However, current mobile platforms include 
only coarse access control mechanisms to protect such data. In this pa¬ 
per, we introduce interaction-based declassification policies, in which the 
user’s interactions with the app constrain the release of sensitive infor¬ 
mation. Our policies are defined extensionally, so as to be independent of 
the app’s implementation, based on sequences of security-relevant events 
that occur in app runs. Policies use LTL formulae to precisely specify 
which secret inputs, read at which times, may be released. We formalize 
a semantic security condition, interaetion-based noninterference, to de¬ 
fine our policies precisely. Finally, we describe a prototype tool that uses 
symbolic execution of Dalvik bytecode to check interaction-based de- 
classification policies for Android, and we show that it enforces policies 
correctly on a set of apps. 

Keywords: Information flow, program analysis, symbolic execution. 


1 Introduction 

The Android platform includes a permission system that aims to prevent apps 
from abusing access to sensitive information, such as contacts and location. Un¬ 
fortunately, once an app is installed, it has carte blanche to use any of its per¬ 
missions in arbitrary ways at run time. For example, an app with location and 
Internet access could continuously broadcast the device’s location, even if such 
behavior is not expected by the user. 

To address this limitation, this paper presents a new framework for Android 
app security based on information flow control [?] and user interactions. The 
key idea behind our framework is that users naturally express their intentions 
about information release as they interact with an app. For example, clicking a 
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AFOSR grants FA9550-12-1-0334 and FA9550-14-1-0334, the partnership between 
UMIACS and the Laboratory for Telecommunication Sciences, and the National 
Security Agency. 
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button may permit an app to release a phone number over the Internet. Or, as 
another example, toggling a radio button from “coarse” to “fine” and back to 
“coarse” may temporarily permit an app to use fine-grained GPS location rather 
than a coarse-grained approximation. 

To model these kinds of scenarios, we introduce interaction-based declassi¬ 
fication policies, which extensionally specify what information flows may occur 
after which sequences of events. Events are GUI interactions (e.g., clicking a 
button), inputs (e.g., reading the phone number), or outputs (e.g., sending over 
the Internet). A policy is a set of declassification conditions, written t> S, 
where ^ is a linear-time temporal logic (LTL) |18j formula over events, and S 
is a sensitivity level. If (j) holds at the time an input occurs, then that input is 
declassified to level S. We formalize a semantic security condition, interaction- 
based noninterference (IBNI), over sets of event traces generated by an app. 
Intuitively, IBNI holds of an app and policy if observational determinism [25) 
holds after all inputs have been declassified according to the policy. (Section]^ 
describes policies further, and Sectionpresents our formal definitions.) 

We introduce GlickRelease, a static analysis tool to check whether an An¬ 
droid app and its declassification policy satisfy IBNI. GlickRelease generates 
event traces using SymDroid [9], a Dalvik bytecode symbolic executor. Glick¬ 
Release works by simulating user interactions with the app and recording the 
resulting execution traces. In practice, it is not feasible to enumerate all program 
traces, so GlickRelease generates traces up to some input depth of n GUI events. 
GlickRelease then synthesizes a set of logical formulae that hold if and only if 
IBNI holds, and uses Z3 [15] to check their satisfiability. (Section describes 
GlickRelease in detail.) 

To validate GlickRelease, we used it to analyze four Android apps, including 
both secure and insecure variants of those apps. We ran each app variant under 
a range of input depths, and confirmed that, as expected, GlickRelease scales 
exponentially. However, we manually examined each app and its policy, and 
found that an input depth of at most 5 is sufficient to guarantee detection of a 
security policy violation (if any) for these cases. We ran GlickRelease at these 
minimum input depths and found that it correctly passes and fails the secure 
and insecure app variants, respectively. Moreover, at these depths, GlickRelease 
takes just a few seconds to run. (Sectiondescribes our experiments.) 

In summary, we believe that GlickRelease takes an important step forward in 
providing powerful new security mechanisms for mobile devices. We expect that 
our approach can also be used in other GUI-based, security-sensitive systems. 

2 Example Apps and Policies 

We begin with two example apps that show interesting aspects of interaction- 
based declassification policies. 

Bump app. The boxed portion of Fig. gives (simplified) source code for an 
Android app that releases a device’s unique ID and/or phone number. This 
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public class BumpApp extends Activity { 
protected void onCreate(...) { 

Button sendBtn = (Button) findViewByld(.. .); 

CheckBox idBox = (CheckBox) findViewByld(...); 

CheckBox phBox = (CheckBox) findViewByld(...); 

TelephonyManager manager = TelephonyManager.getTelephonyManager(); 
final int id = manager.getDeviceld(); 
final int ph = manager.getPhoneNumber(); 
idBox.setChecked(false); phBox.setChecked(false); 
sendBtn. setOnClickListener ( 
new OnClickListener() { 
public void onClick(View v) { 
if (idBox.isCheckedO) 

Internet .sendlnt(id ); //Internet.sendlnt(ph); 
if (phBox.isCheckedO) 

Internet . sendlnt(ph); //Internet.sendlnt(id); 

}})}} 


id! * A(.^(s0ndBtnIunit A /cisi^idBox, tru6))) Lovj^ 
ph! * A(7^(sendBtn!unit A Zasf(phBox, true))) > Low 


Fig. 1. “Bump” app and policy. 


app is inspired by the Bump app, which let users tap phones to share selected 
information with each other. We have interspersed an insecure variant of the app 
in the red code on lines [T^ and [T^ which we will discuss in Section |3.1[ 

Each screen of an Android app is implemented using a class that extends 
Activity. When an app is launched, Android invokes the onCreate method for a 
designated main activity. (This is part of the activity lifecycle [H], which includes 
several methods called in a certain order. For this simple app, and the other apps 
used in this paper, we only need a single activity with this one lifecycle method.) 
That method retrieves (lines [3]|^ the GUI IDs of a button (marked “send”) and 
two checkboxes (marked “ID” and “phone”). The onCreate method next gets 
an instance of the TelephonyManager, uses it to retrieve the device’s unique ID 
and phone number information, and unchecks the two checkboxes as a default. 
Then it creates a new callback (line[IT|) to be invoked when the “send” button is 
clicked. When called, that callback releases the user’s ID and/or phone number, 
depending on the checkboxes. 

This app is written to work with ClickRelease, a symbolic execution tool we 
built to check whether apps satisfy interaction-based declassification policies. As 
we discuss further in Section]^ ClickRelease uses an executable model of Android 
that abstracts away some details that are unimportant with respect to security. 
While a real app would release information by sending it to a web server, here 
we instead call a method Internet.send Int. Additionally, while real apps include 
an XML file specifying the screen layout of buttons, checkboxes, and so on, 
ClickRelease creates those GUI elements on demand at calls to findViewByld 
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(since their screen locations are unimportant). Finally, we model the ID and 
phone number as integers to keep the analysis simpler. 

ClickRelease symbolically executes paths through subject apps, recording a 
trace of events that correspond to certain method calls. For example, one path 
through this app generates a trace 

id!42, ph!43, idBoxItrue, sendBtnIunit, netout!42 

Each event has a name and a value. Here we have used names id and ph for 
secret inputs, id Box and sendBtn for GUI inputs, and netout for the network 
send. In particular, the trace above indicates 42 is read as the ID, 43 is read 
as the phone number, the ID checkbox is selected, the send button is clicked 
(carrying no value, indicated by unit), and then 42 is sent on the network. In 
ClickRelease, events are generated by calling certain methods that are specially 
recognized. For example, ClickRelease implements the manager.getDeviceld call 
as both returning a value and emitting an event. 

Notice here that in the trace, callbacks to methods such as id Box and sendBtn 
correspond to user interactions. The key idea behind our framework is that these 
actions convey the user’s intent as to which information should be released. More¬ 
over, traces also contain actions relevant to information release—here the reads 
of the ID and phone number, and the network send. Thus, putting both user 
interactions and security-sensitive operations together in a single trace allows 
our policies to enforce the user’s intent. 

The policy for this example app is shown at the bottom of Fig.[^ Policies are 
comprised of a set of declassification conditions of the form (j)0 S, where (j) is an 
LTL formula describing event traces and 5" is a security level. Such a condition 
is read, “At any input event, if (f holds at that position of the event trace, then 
that input is declassified at level S'.” For this app there are two declassihcation 
conditions. The top condition declassifies (to Low) an input that is a read of 
the ID at any value (indicated by *), if sometime in the future (indicated by 
the T modality) the send button is clicked and, when that button is clicked, the 
last value of the ID checkbox was true. (Note that last is not primitive, but is 
a macro that can be expanded into regular LTL.) The second declassihcation 
condition does the analogous thing for the phone number. 

To check such a policy, ClickRelease symbolic executes the program, generat¬ 
ing per-path traces; determines the classihcation level of every input; and checks 
that every pair of traces satishes noninterference. Note that using LTL provides 
a very general and expressive way to describe the sequences of events that imply 
declassihcation. For example, here we precisely capture that only the last value 
of the checkbox matters for declassihcation. For example, if a user selects the ID 
checkbox but then unselects it and clicks send, the ID may not be released. 

Although this example relies on a direct how, ClickRelease can also detect 
implicit hows. Section [3. 2| dehnes an appropriate version of noninterference, and 
the experiments in Section include a subject program with an implicit how. 

Notice this policy depends on the app reading the ID and phone number 
when the app starts. If the app instead waited until after the send button were 
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public class ToggleRes extends Activity { ... 

LocSharer mLocSharer = new LocSharer(); 

RadioManager mRadio = new RadioManager(); 
protected void onCreate(...) { ...} 

private class LocSharer implements LocationListener { ... 
public LocSharer(RadioManager rm) { 

Im = (LocationManager) getSystemService(LOCATION_SERVICE); 
lm.requestLocationUpdates(mCurrentProvider, SHARE_INTERVAL, distance, this); 

} 

public void onLocationChanged(Location I) { 
if (mRadio.mFine) { 

Internet . sendlnt( I . mLatitude); 

Internet . sendlnt( I . mLongitude); 

} else { 

Internet .sendlnt(l .mLatitude & 0 xffffffOO ); 

Internet .sendlnt(l .mLongitude & OxffffffOO ); 

} } } 

private class RadioManager 
implements OnClickListener { 
public boolean mFine = false; 
public void onClick(View v) { mFine = ImFine; } 

H_ 

longitude! * AZasi(mRadio, true) > Low, 
longitude! * AZasi(mRadio, false) > MaskLowerS 

Fig. 2. Location sharing app and policy. 


clicked, it would violate this policy. We could address this by replacing the J- 
modality by V (past) in the policy, and we could form a disjunction of the two 
policies if we wanted to allow either implementation. More generally, we designed 
our framework to be sensitive to such choices to support reasoning about secret 
values that change over time. We will see an example next. 

Location resolution toggle app. Fig. gives code for an app that shares location 
information, either at full or truncated resolution depending on a radio button 
setting. The app’s onCreate method displays a radio button (code not shown) and 
then creates and registers a new instance of RadioManager to be called each time 
the radio button is changed. That class maintains field mFine as true when the 
radio button is set to full resolution and false when set to truncated resolution. 

Separately, onCreate registers LocSharer to be called periodically with the 
current location. It requests location updates by registering a callback with the 
LocationManager system service. When called, LocSharer releases the location, 
either at full resolution or with the lower 8 bits masked, depending on mFine. 

The declassification policy for longitude appears below the code; the policy 
for latitude is analogous. This policy allows the precise longitude to be released 
when mRadio is set to fine, but only the lower eight bits to be released if mRadio 
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Primitives 

p ::= n \ true | false | unit | /(pi,... ,Pi) 

Events 

rj ■.:= namelp 

Traces 

t ::= 77 list 


(a) Event and Trace Definitions. 

Policies 

P -.-01,02,... 

Conditions 

G ::= S 

Security Levels 

S ::= High \ Low \ MaskLowerS | ... 

Atoms 

A namels | s © s 

Messages 

s x\p\* 

Formulae 

0 ::= A 1 -■(/) \ cj) A (/> \ (f) y cf) \ (f) ^ (/> \ 3x.4> \ 'ix.tj) 


\X4>\<tiU 4>\Q<f)\T4>\<l>s (i>\V4> 


(b) Interaction-based Declassification Policy Language. 


Fig. 3. Formal definitions. 


is set to coarse. Here ClickRelease knows that at the MaskLowerS level, it should 
consider outputs to be equivalent up to differences in the lower 8 bits. 

Finally, notice that this policy does not use the future modality. This is 
deliberate, because location may be read multiple times during the execution, at 
multiple values, and the security level of those locations should depend on the 
state of the radio button at that time. For example, consider a trace 

mRadiolfalse, longitudeRi, mRadioltrue, longitude!v 2 

The second declassification condition (longitude!*AZast(mRadio, false)) will match 
the event with vi, since the last value of mRadio was false, and thus vi may be 
declassified only to MaskLowerS. Whereas the first declassification condition will 
match the event with V 2 , hence it may be declassified to Low. 

3 Program Traces and Security Definition 

Next, we formally define when a set of program traces satisfies an interaction- 
based declassification policy. 

3.1 Program Traces 

Fig. [^a) gives the formal syntax of events and traces. Primitives p are terms 
that can be carried by events, e.g., values for GUI events, secret inputs, or 
network sends. In our formalism, primitives are integers, booleans, and terms 
constructed from primitives using uninterpreted constructors /. As programs 
execute, they produce a trace t of events ry, where each event namelp pairs an 
event name name with a primitive p. We assume event names are partitioned into 
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those corresponding to inputs and those corresponding to outputs. For all the 
examples in this paper, all names are inputs except netout, which is an output. 

Due to space limitations, we omit details of how traces are generated. These 
details, along with definition of our LTL formulas, can be found in appendices [A| 
an d|l respectively. Instead, we simply assume there exists some set T containing 
all possible traces a given program may generate. For example, consider the 
insecure variant bump app in Fig. which replaces the black code with the 
red code on lines lines and |16| This app sends the phone number when the 
email box is checked and vice-versa. Thus, its set T contains, among others, the 
following two traces: 

id!0, ph!0, idBoxItrue, sendBtnIunit, netoutIO (1) 
id!0, ph!l, idBoxItrue, sendBtnIunit, netoutll (2) 

In the first trace, ID and phone number are read as 0, the ID checkbox is selected, 
the button is clicked, and 0 is sent. The second trace is similar, except the phone 
number and sent value are 1. Below, we use these traces to show this program 
violates its security policy. 


3.2 Interaction-based Declassification Policies 

We now define our policy language precisely. Fig. ib) gives the formal syntax of 
declassification policies. A policy P is a set of declassification conditions Ci of the 
form where fi is an LTL formula describing when an input is declassified, 

and Si is a security level at which the value in that event is declassified. 

As is standard, security levels S form a lattice. For our framework, we require 
that this lattice be finite. We include High and Low security levels, and we can 
generalize to arbitrary lattices in a straightforward way. Here we include the 
MaskLowerS level from Fig.j^as an example, where Low D MaskLowerS D High. 
Note that although we include High in the language, in practice there is no reason 
to declassify something to level High, since then it remains secret. 

The atomic predicates A of LTL formulae match events, e.g., atomic predi¬ 
cate namelp matches exactly that event. We include * for matches to arbitrary 
primitives. We allow event values to be variables that are bound in an enclosing 
quantifier. The atomic predicates also include atomic arithmetic statements; here 
0 ranges over standard operations such as +, <, etc. The combination of these 
lets us describe complex events. For example, we could write 3x.spinnerlx f\x > 2 
to indicate the spinner was selected with a value greater than 2. 

Atomic predicates are combined with the usual boolean connectives (-i. A, 
V, —>■) and existential and universal quantification. Formulae include standard 
LTL modalities X (next), U (until), Q (always), X (future), f S (since), and 
Vf (past). We include a wide range of modalities, rather than a minimal set, 
to make policies easier to write. Formulae also include last{name, p), which is 
syntactic sugar for ^(namel*) S namelp. We assume a standard interpretation 
of LTL formulae over traces m- We write t,i \= f if trace t is a model of (j) at 
position i in the trace. 


Next consider a trace t £ T for an arbitrary program. We write level{t, P, i) 
for the security level that policy P assigns to the event t[i\-. 

kveHt.P.,) = I M h ([.] = 

\Low t[i] = netoutip 

In other words, for inputs, we take the greatest lower bound (the most de¬ 
classified) of the levels from all declassification conditions that apply. We always 
consider network outputs to be declassified. Notice that if no policy applies, the 
level is H by definition of greatest lower bound. 

For example, consider trace (1) above with respect to the policy in Fig.[^ At 
position 0, the LTL formula holds because the ID box is eventually checked and 
then the send button is clicked, so level{{l), P, 0) = Low. However, level{{l),P, 1) = 
High because no declassification condition applies for ph (ph Box is never checked). 
And /e?;e/((I), P, 4) = Low, because that position is a network send. 

Next consider applying this definition to the GUI inputs. As written, we 
have level{{l),P,2) = level{{l), P,3) = High. However, our app is designed to 
leak these inputs. For example, an adversary will learn the state of id Box if they 
receive a message with an ID. Thus, for all the subject apps in this paper, we also 
declassify all GUI inputs as Low. For the example in Fig. this means adding 
the conditions idBox! * \>Low, phBox! * \>Low, and sendBtn! * i>Low. In general, 
the security policy designer should decide the security level of GUI inputs. 

Next, we can apply level pointwise across a trace and discard any trace 
elements that are below a given level S. We define 

I T otherwise 

We write level{t, p)'®’*"' for the same filtering, except output events (i.e., network 
sends) are removed as well. Considering the traces (I) and (2) again, we have 

/ei'e/((I), P)^°™=id!0, idBoxItrue, sendBtnlunit, netoutIO 
/ei'e/((2), P)^°™=id!0, idBoxItrue, sendBtnIunit, netoutll 
level{{l), P)'^'”"’™=id!0, idBoxItrue, sendBtnIunit 
level({2), P)'^'”"’™=idl0, idBoxItrue, sendBtnIunit 

Finally, we can define a program to satisfy noninterference if, for every pair 
of traces such that the inputs at level S are the same, the outputs at level S are 
also the same. To account for generalized lattice levels such as MaskLowerS, we 
also need to treat events that are equivalent at a certain level as the same. For 
example, at MaskLowerS, outputs Oxffffffff and OxffffffOO are the same, 
since they do not differ in the upper 24 bits. Thus, we assume for each security 
level S there is a appropriate equivalence relation =s, e.g., for MaskLowerS, it 
compares elements ignoring their lower 8 bits. Note that x =low y is simply 
X = y and x =High y is always true. 
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Definition 1 (Interaction-based Noninterference (IBNI)). A program sat¬ 
isfies security policy P, if for all S and for all ti,t 2 & T (the set of traces of the 
program) the following holds: 

level{ti,Pf’^’^=slevel{t 2 ,Pf'"’^ level{ti,Pf =S level{t 2 ,Pf 

Looking at traces for the insecure app, we see they violate non-interference, 
because level{{l), = level{{2), p)Lon,,in^ level{{l), 7 ^ level{f2), 

(they differ in the output). We note that our definition of noninterference makes 
it a 2 -hypersafety property [SI?]- 


4 Implementation 


We built a prototype tool, ClickRelease, to check whether Android apps obey 
the interaction-based declassification policies described in Section [^ ClickRe¬ 
lease is based on SymDroid |S], a symbolic executor for Dalvik bytecode, which 
is the bytecode format to which Android apps are compiled. As is standard, Sym¬ 
Droid computes with symbolic expressions that may contain symbolic variables 
representing sets of values. At conditional branches that depend on symbolic 
variables, SymDroid invokes Z3 m to determine whether one or both branches 
are feasible. As it follows branches, SymDroid extends the current path condition^ 
which tracks branches taken so far, and forks execution when multiple paths are 
possible. Cadar and Sen [T] describe symbolic execution in more detail. 

SymDroid uses the features of symbolic execution to implement nondetermin- 
istic event inputs (such as button clicks or spinner selections), up to a certain 
bound. Since we have symbolic variables available, we also use them to represent 


arbitrary secret inputs, as discussed below in Sec. 4.2 There are several issues 
that arise in applying SymDroid to checking our policies, as we discuss next. 


4.1 Driving App Execution 

Android apps use the Android framework’s API, which includes classes for re¬ 
sponding to events via callbacks. We could try to account for these callbacks 
by symbolically execution Android framework code directly, but past experience 
suggests this is intractable: the framework is large, complicated, and includes na¬ 
tive code. Instead, we created an executable model, written in Java, that mimics 
key portions of Android needed by our subject apps. Our Android model includes 
facilities for generating clicks and other GUI events (such as the View, Button, 
and CheckBox classes, among others). It also includes code for Location Manager, 
TelephonyManager, and other basic Android classes. 

In addition to code modeling Android, the model also includes simplified ver¬ 
sions of Java library classes such as StringBuffer and StringBuilder. Our versions 
of these APIs implement unoptimized versions of methods in Java and escape to 
internal SymDroid functions to handle operations that would be unduly complex 
to symbolically execute. For instance, SymDroid represents Java String objects 
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with OCaml strings instead of Java arrays of characters. It thus models methods 
such as String.concat with internal calls to OCaml string manipulation functions. 
Likewise, reflective methods such as Class.getName are handled internally. 

For each app, we created a driver that uses our Android model to simulate 
user input to the GUI. The driver is specific to the app since it depends on 
the app’s GUI. The driver begins by calling the app’s onCreate method. Next it 
invokes special methods in the Android model to inject GUI events. There is one 
such method for each type of GUI element, e.g., buttons, checkboxes, etc. For 
example. Trace.addClick(id) generates a click event for the given id and then calls 
the appropriate event handler. The trace entry contains the event name for that 
kind of element, and a value if necessary. Event handlers are those that the app 
registered through standard Android framework mechanisms, e.g., in onCreate. 

Let m be the number of possible GUI events. To simulate one arbitrary 
GUI event, the driver uses a block that branches m ways on a fresh symbolic 
variable, with a different GUI action in each branch. Typical Android apps never 
exit unless the framework kills them, and thus we explore sequences of events 
only up to a user-specified input depth n. Thus, in total, the driver will execute 
at least m" paths. 


4.2 Symbolic Variables in Traces 

In addition to GUI inputs, apps also use secret inputs. We could use SymDroid to 
generate concrete secret inputs, but instead we opt to use a fresh symbolic vari¬ 
able for each secret input. For example, the call to manager.getDeviceld in Fig.[^ 
returns a symbolic variable, and the same for the call to manager.getPhoneNumber. 
This choice makes checking policies using symbolic execution a bit more power¬ 
ful, since, e.g., a symbolic integer variable represents an arbitrary 32-bit integer. 
Note that whenever GlickRelease generates a symbolic variable for a secret input, 
it also generates a trace event corresponding to the input. 

Recall that secret inputs may appear in traces, and thus traces may now 
contain symbolic variables. For example, using ai ’s as symbolic variables for the 
secret ID and phone number inputs, the traces (1) and (2) become 

idloi, ph!a 2 , idBoxItrue, sendBtnIunit, netout!a 2 (10 
idloi, ph!Q; 2 ; idBoxItrue, sendBtnIunit, netoutla 2 (20 

We must take care when symbolic variables are in traces. Recall level checks 
t,i \= (j) and then assigns a security level to position i. If (j) depends on symbolic 
variables in t, we may not be able to decide this. For example, if the third 
element in (10 were idBoxIaa, then we would need to reason with conditional 
security levels such as level{t, P, 0) = if 03 then Low else High. We avoid the 
need for such reasoning by only using symbolic variables for secret inputs, and 
by ensuring the level assigned by a policy does not depend on the value of a 
secret input. We leave supporting more complex reasoning to future work. 
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4.3 Checking Policies with Z3 

Each path explored by SymDroid yields a pair where t is the trace and 

is the path condition. ClickRelease uses Z3 to check whether a given set of 
such trace-path condition pairs satisfies a policy P. Recall that Definition 
assumes for each S there is an =5 relation on traces. We use the same relation 
below, encoding it as an SMT formula. For our example lattice, =High produces 
true, =Low produces a conjunction of equality tests among corresponding trace 
elements, and = MaskLowerS produces the conjunction of equality tests of the 
bitwise-and of every element with OxffffffOO. 

Given a trace t, let t' be t with its symbolic variables primed, so that the 
symbolic variables of t and t' are disjoint. Given a path condition <!>, define <1>' 
similarly. Now we can give the algorithm for checking a security policy. 

Algorithm 1 To check a set T of trace-path condition pairs, do the following. 
Let P be the app’s security policy. Apply level across each trace to obtain the 
level of each event. For each and (^ 2 ,^ 2 ) inT x T, and for each S, ask 

Z3 whether the following formula (the negation of Definition^ is unsatisfiable: 

level{ti,Pf'^’^ =S level{t' 2 , A level{ti,Pf levelft'^, Pf A ^>i A 

If no such formula is unsatisfiable, then the program satisfies noninterference. 

We include <Li and ^2 to constrain the symbolic variables in the trace. More 
precisely, ti represents a set of concrete traces in which its symbolic variables 
are instantiated in all ways that satisfy <?i, and analogously for 

If the above algorithm finds an unsatisfiable formula, then Z3 returns a coun¬ 
terexample, which SymDroid uses in turn to generate a pair of concrete traces as 
a counterexample. For example, consider traces (!’) and (2’) above, and prime 
symbolic variables in (2’). Those traces have the trivial path condition true, since 
neither branches on a symbolic input. Thus, the formula passed to Z3 will be: 

ai = a'^Atrue = trueAunit = unitA (oii a^Vtrue 7^ trueVunit 7^ unitVa2 7^ 02) 

Thus we can see a satisfying assignment with a\ = aj and 0 x 2 02 ! hence 

noninterference is violated. 

4.4 Minimizing Calls to Z3 

A naive implementation of the noninterference check generates rif equations, 
where n is the number of traces produced by ClickRelease to be checked by Z3. 
However, we observed that many of these equations correspond to pairs of traces 
with different sequences of GUI events. Since GUI events are low inputs in all 
our policies, these pairs trivially satisfy noninterference (the left-hand side of the 
implication in Definitionis false). Thus, we need not send those equations to 
Z3 for an (expensive) noninterference check. 

We exploit this observation by organizing SymDroid’s output traces into a 
tree, where each node represents an event, with the initial state at the root. 
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Traces with common prefixes share the same ancestor traces in the tree. We 
systematically traverse this tree using a cursor ti, starting from the root. When 
ti reaches a new input event, we then traverse the tree using another cursor t 2 , 
also starting from the root. As ^2 visits the tree, we do not invoke Z3 on any 
traces with fewer input events than ti (since they are not low-equivalent to ti). 
We also skip any subtrees where input events differ. 


5 Experiments 

To evaluate ClickRelease, we ran it on four apps, including the two described in 

Section We also ran ClickRelease on several insecure variants of each app, to 

ensure it can detect the policy violations. The apps and their variants are: 

— Bump. The bump app and its policy appear in Fig. The first insecure 
variant counts clicks to the send button sends the value of the ID after three 
clicks, regardless of the state of the ID checkbox. The second (indicated in 
the comments in the program text) swaps the released information—if the ID 
box is checked, it releases the phone number, and vice-versa. 

— Location toggle. The location toggle app and its policy appear in Fig. The 
first insecure variant always shares fine-grained location information, regard¬ 
less of the radio button setting. The second checks if coarse-grain information 
is selected. If so, it stores the fine-grained location (but does not send it yet). 
If later the fine-grained radio button is selected, it sends the stored location. 
Recall this is forbidden by the app’s security policy, which allows the release 
only of locations received while the fine-grained option is set. 

— Contact picker. We developed a contact picker app that asks the user to select 
a contact from a spinner and then click a send button to release the selected 
contact information over the network. The security policy for this app requires 
that no contact information leaks unless it is the last contact selected before 
the button click. (For example, if the user selects contact I, selects contact 2, 
and then clicks the button, only contact 2 may be released.) Note that since 
an arbitrarily sized list of contacts would be difficult for symbolic execution 
(since then there would be an unbounded number of ways to select a contact), 
we limit the app to a fixed set of three contacts. The first insecure variant of 
this app scans the set of contacts for a specific one. If found, it sends a message 
revealing that contact exists before sending the actual selected contact. The 
second insecure variant sends a different contact than was selected. 

— WhereRU. Lastly, we developed an app that takes push requests for the user’s 
location and shares it depending on user-controlled settings. The app contains 
a radio group with three buttons, “Share Always,” “Share Never,” and “Share 
On Click.” There is also a “Share Now” button that is enabled when the “Share 
On Click” radio button is selected. When a push request arrives, the security 
policy allows sharing if (I) the “Always” button is selected, or (2) the “On 
Click” button is selected and the user presses “Share Now.” Note that, in the 
second case, the location may change between the time the request arrives 
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Fig. 4. Runtime vs. number of events. 


and the time the user authorizes sharing; the location to be shared is the 
one in effect when the user authorized sharing, i.e., the one from the most 
recent location update before the button click. Also, rather than include the 
full Android push request API in our model, we simulated it using a basic 
callback. This app has two insecure variants. In the first one, when the user 
presses the “Share Now” button, the app begins continuously sharing (instead 
of simply sharing the single location captured on the button press). In the 
second, the app shares the location immediately in response to all requests. 


Scalability. We ran our experiments on a 4-core i7 CPU @3.5GHz with 16GB 
RAM running Ubuntu 14. For each experiment we report the median of 10 runs. 

In our first set of experiments, we measured how ClickRelease’s performance 
varies with input depth. Figure shows running time (log scale) versus input 
depth for all programs and variants. For each app, we ran to the highest input 
depth that completed in one hour. 
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App 

Input 

Depth 

Time (ms) 

Exploration 

Analysis 

Total 

Bump 

3 

114 

15 

142 

Bump (insecure 1) 

5 

2,100 

1,577 

3,690 

Bump (insecure 2) 

4 

266 

70 

344 

Location toggle 

2 

113 

12 

128 

Location toggle (insecure 1) 

2 

143 

12 

163 

Location toggle (insecure 2) 

3 

117 

12 

143 

Contact picker 

2 

79 

2 

94 

Contact picker (insecure 1) 

2 

325 

27 

361 

Contact picker (insecure 2) 

2 

149 

9 

170 

WhereRU 

3 

849 

183 

1,045 

WhereRU (insecure 1) 

3 

860 

234 

1,108 

WhereRU (insecure 2) 

2 

257 

10 

280 


Fig. 5. Results at minimum input depth. 


For each app, we see that running time grows exponentially, as expected. 
The maximum input depth before timeout (i.e., where each curve ends) ranges 
from five to nine. The differences have to do with the number of possible events 
at each input point. For example, WhereRU has seven possible input events, so 
it has the largest possible “fan out” and times out with an input depth of five. 
In contrast. Bump and Location Toggle have just three input events and time 
out with an input depth of nine. Notice also the first insecure variant of Contact 
Picker times out after fewer events than the other variants. Investigating further, 
this occurs due to that app’s implicit flow (recall the app branches on the value 
of a secret input). Implicit flows cause symbolic execution to take additional 
branches depending on the (symbolic) secret value. 

Minimum Input Depth. Next, for each variant, we manually calculated a min¬ 
imum input depth guaranteed to find a policy violation. To do so, first we de¬ 
termined possible app GUI states. For example, in Bump (Fig. [^, there is a 
state with id Box and phBox both checked, a state with just id Box checked, etc. 
Then we examined the policy and recognized that certain input sequences lead 
to equivalent states modulo the policy. For example, input sequences that click 
id Box an even number of times and then click send are all equivalent. Full anal¬ 
ysis reveals that an input depth of three (which allows the checkboxes to be set 
any possible way followed by a button click) is sufficient to reach all possible 
states for this policy. We performed similar analysis on other apps and variants. 

Fig. [5] summarizes the results of running with the minimum input depth for 
each variant, with the depths listed in the second column. We confirmed that, 
when run with this input depth, ClickRelease correctly reports the benign app 
variants as secure and the other app variants as insecure. The remaining columns 
of Fig. [^report ClickRelease’s running time (in milliseconds), broken down by 
the exploration phase (where SymDroid generates the set of symbolic traces) and 
the analysis phase (where SymDroid forms equations about this set and checks 
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them using Z3). Looking at the breakdown between exploration and analysis, we 
see that the former dominates the running time, i.e., most of the time is spent 
simply exploring program executions. We see the total running time is typically 
around a second or less, while for the first insecure variant of Bump it is closer 
to 4 seconds, since it uses the highest input depth. 

Our results show that while ClickRelease indeed scales exponentially, to ac¬ 
tually find security policy violations we need only run it with a low input depth, 
which takes only a small amount of time. 


6 Limitations and Future Work 


There are several limitations of ClickRelease we plan to address in future work. 

Thus far we have applied ClickRelease to a set of small apps that we de¬ 
veloped. There are two main engineering challenges in applying ClickRelease to 


other apps. First, our model of Android (Section 4.1) only includes part of the 
framework. To run on other apps, it will need to be expanded with more Android 
APIs. Second, we speculate that larger apps may require longer input depths to 
go from app launch to interfering outputs. In these cases, we may be able to 
start symbolic execution “in the middle” of an app (e.g., as in the work of Ma 
et al. [13]) to skip uninteresting prefixes of input events. 

ClickRelease also has several limitations related to its policy language. First, 
ClickRelease policies are fairly low level. Complex policies—e.g., in which clicking 
a certain button releases multiple pieces of information—can be expressed, but 
are not very concise. We expect as we gain more experience writing ClickRelease 
policies, we will discover useful idioms that should be incorporated into the policy 
language. Similarly, situations where several methods in sequence operate on and 
send information should be supported. Second, currently ClickRelease assumes 
there is a single adversary who watches netout. It should be straightforward to 
generalize to multiple output channels and multiple observers, e.g., to model 
inter-app communication. Third, we do not consider deception by apps, e.g., we 
assume the policy writer knows whether the sendBtn is labeled appropriately as 
“send” rather than as “exit.” We leave looking for such deceptive practices to 
future work. 

Finally, since ClickRelease explores a limited number of program paths it is 
not sound, i.e., it cannot guarantee the absence of policy violations in general. 
However, in our experiments we were able to manually analyze apps to show 
that exploration up to a certain input depth was sufficient for particular apps, 
and we plan to investigate generalizing this technique in future work. 


7 Related Work 

ClickRelease is the first system to enforce extensional declassification policies in 
Android apps. It builds on a rich history of research in usable security, informa¬ 
tion flow, and declassification. 
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One of the key ideas in ClickRelease is that GUI interactions indicate the se¬ 
curity desires of users. Roesner et al. [20] similarly propose access control gadgets 
(ACGs), which are GUI elements that, when users interact with them, grant per¬ 
missions. Thus, AGGs and GlickRelease both aim to better align security with 
usability [21]. GlickRelease addresses secure information flow, especially propa¬ 
gation of information after its release, whereas AGGs address only access control. 

Android-based systems. TaintDroid |7] is a run-time information-flow tracking 
system for Android. It monitors the usage of sensitive information and detects 
when that information is sent over insecure channels. Unlike GlickRelease, Taint- 
Droid does not detect implicit flows. 

Appintent |23j uses symbolic execution to derive the context, meaning in¬ 
puts and GUI interactions, that causes sensitive information to be released in an 
Android app. A human analyst examines that context and makes an expert judg¬ 
ment as to whether the release is a security violation. ClickRelease instead uses 
human-written LTL formulae to specify whether declassifications are permitted. 
It is unclear from [22] whether Appintent detects implicit flows. 

Pegasus [2] combines static analysis, model checking, and run-time monitor¬ 
ing to check whether an app uses API calls and privileges consistently with users’ 
expectations. Those expectations are expressed using LTL formulae, similarly to 
ClickRelease. Pegasus synthesizes a kind of automaton called a permission event 
graph from the app’s bytecode then checks whether that automaton is a model 
for the formulae. Unlike ClickRelease, Pegasus does not address information flow. 

Jia et al. [10] present a system, inspired by Flume [12] . for run-time enforce¬ 
ment of information flow policies at the granularity of Android components and 
apps. Their system allows components and apps to perform trust declassifica¬ 
tion according to capabilities granted to them in security labels. In contrast, 
ClickRelease reasons about declassification in terms of user interactions. 

Security type systems Security type systems [?] statically disallow programs 
that would leak information. O’Neill et al. m and Clark and Hunt [5] define 
interactive variants of noninterference and present security type systems that 
are sound with respect to these definitions. 

Integrating declassification with security type systems has been the focus of 
much research. Chong and Myers [3] introduce declassification policies that con¬ 
ditionally downgrade security labels. Their policies use classical propositional 
logic for the conditions. ClickRelease can be seen as providing a more expres¬ 
sive language for conditions by using LTL to express formulae over events. SIF 
(Servlet Information Flow) [3] is a framework for building Java servlets with 
information-flow control. Information managed by the servlet is annotated in 
the source code with security labels, and the compiler ensures that information 
propagates in ways that are consistent with those labels. The SIF compiler is 
based on Jif an information-flow variant of Java. 

All of these systems require adding type annotations to terms in the pro¬ 
gram code, e.g., method parameters, etc. In contrast, ClickRelease policies are 
described in terms of app inputs and outputs. 
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Event Based Models and Declassification Vaughan and Chong [22] define ex¬ 
pressive declassification policies that allow functions of secret information to be 
released after events occur, and extend the Jif compiler to infer events. ClickRe- 
lease instead ties events to user interactions. 

Rafnsson et al. m investigate models, definitions, and enforcement tech¬ 
niques for secure information flow in interactive programs in a purely theoretical 
setting. Sabelfeld and Sands [H] survey approaches to secure declassification in a 
language-based setting. ClickRelease can be seen as addressing their “what” and 
“when” axes of declassification goals: users of Android apps interact with the 
GUI to control when information may be released, and the GUI is responsible 
for conveying to the user what information will be released. 


8 Conclusion 

We introduced interaction-based declassification policies, which describe what 
and when information can flow. Policies are defined using LTL formulae describ¬ 
ing event traces, where events include GUI actions, secret inputs, and network 
sends. We formalized our policies using a trace-based model of apps based on 
security relevant events. Finally, we described GlickRelease, which uses sym¬ 
bolic execution to check interaction-based declassification policies on Android, 
and showed that GlickRelease correctly enforces policies on four apps, with one 
secure and two insecure variants each. 
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A Operational Semantics / Trace Generation 

To illustrate how traces are generated, we introduce an operational semantics 
for a model of Android programs. Our semantics uses an abstract machine which 
transitions between states and reads messages on a message queue. A state is a 
tuple {M,a,H) that includes a message queue M, which is a list of pairs of a 
channel and a primitive; a heap tr, which maps locations to values; and a handler 
map H, which maps channel names to functions installed to handle events on 
those channels. Each channel may have at most one handler. We write S.X 
(where X could be M, cr, or H), to mean the X component of E. Similarly, we 
write S[X ^ X'] to mean E with the X component replaced by X'. 

As programs execute, they produce a trace t of events rj. Events are writes, 
written namelp, of primitive p from channel name. We also include an empty 
event r, which is the identity of trace concatenation. 

Our semantics is stratified into two levels: a big-step semantics, shown at the 
top of Fig. which models evaluation of code in a handler, and a small-step 
semantics, shown at the bottom of the figure, which models the message queue. 
The semantics generates sets of traces containing input and output messages. 

The big-step semantics proves judgments of the form e, Ei IJ. w, A' 2 , meaning 
evaluation of expression e in state Ei produces a value v and new state E 2 . 
The first several rules are standard. RVal evaluates a value to itself, without 
changing the state. RApp evaluates ei to a lambda, evaluates 62 to a value, and 
then evaluates the body of the lambda with the actual argument substituted 
for the formal variable: we use the notation e{x u} for e when unbound 
occurrences of cc in e have been syntactically replaced with v. RRef evaluates e 
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Fig. 6. Semantics for our Android subset. 


to a value v, finds a fresh location £ in the heap, and then evaluates to £, returning 
a state where the heap maps t to v. RAssign evaluates ci to a location £ and 
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then updates the contents of i. RDeref evaluates e to a location and returns 
the contents of that location. 

RIfTrue evaluates ei to a value and if it evaluates to true, evaluates 62 - 
RIfFalse is analogous. ROp evaluates a binary operation by applying the des¬ 
ignated operation to the values of the two subexpressions. 

RCstr and RProj construct terms and project from constructed terms. 
Finally, RInst evaluates ei to a lambda and adds it as a handler for channel 
name] the result value is the unit value unit (a nullary constructor). RSend 
evaluates e to a primitive p, and then adds the message {name,p) to the end 
of the message queue. Here we use @ for concatenation. Note that RSend only 
allows primitives to be sent, and not locations or lambda expressions. 

The first three rules in small-step semantics at the bottom of Fig. prove 
judgments of the form Si — S 2 , meaning the machine can take a step from 
state Si to a new state S 2 , producing an event rj. THandle consumes a message 
{name,p) from the front of the message queue (recall RSend adds a message to 
the end of the message queue), looks up the handler for name, and then invokes 
the handler, passing p as its argument. Running a handler is not an externally 
visible operation, which is indicated by an empty event r on the reduction arrow. 

TInput models an input message, which may be due to user input (e.g., GUI 
clicks) or secret input from the system (e.g., a callback with updated location 
information). This rule non-deterministically picks some channel name and an 
arbitrary primitive p, and then sends p on that channel. The input is recorded 
as an event on the reduction arrow. Notice that here we do not distinguish the 
security level of an input—we choose to leave that up to the security policy 
designer, who can opt to either always designate GUI inputs as low-security (as 
we do in our experiments) or make them high-security. 

TOutput models writes to the network, consuming a message from a dis¬ 
tinguished channel netout. Writing to this channel corresponds to the InfoS- 
ender.sendlnt calls in Sectionj^ Since these messages may be seen by the observer— 
i.e., they are “low visible” outputs—we record the write event on the reduction 
arrow. Note that by convention there is no user handler for this channel. 

Finally, TProg proves a judgment of the form \- e t, meaning running 
the program e produces a trace t of events. This rule creates an initial state Sq 
in which e is bound as a handler on a special onCreate channel, and the message 
queue contains an initial message on that channel. The rule then repeatedly steps 
to the next state n-\-I times. It produces the event trace ?7o ' ’' Notice that 
the length of the trace n is nondeterministic; in general, since these are reactive 
programs, they can usually run for an any number of steps as long as additional 
input arrives. 

From the set of all executions of a program we can extract a set of traces, 
which we later use to define noninterference. 

Definition 2 (Program Traces). We define the set of traces of a program e 
as 

traces{e) = {t | h e~^ t} 
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t, i 1 = name\pi 
t,i \= name\* 
t,i \= Pi ®P2 
t, i 1 = -.0 
t,i\= 4> Alp 
t,i\= (py ip 
t,i\=(p^'ip 
t,i\= Q(p 
t,i \= J-(j> 
t, i \= V(p 
t,i\= <pU tp 

t,i \= (p S ip 

t, i 1 = yx.(p 
t, i 1 = 3x.(p 



t[i] = name\pi 

there exists p,t[i] = name\pi 
\=Pl®P2 
(p 

t,i\= (p and t,i\= ip 
t,i\= (p or t,i\= ip 
t,i\= %p or t, i 1 = -Ip 
for all j . j >i^ (t,j 1= p) 
there exists j,j > i ^ {t,j \= p) 
there exists j, j < i => {t, j ^ p) 
there exists j,j > i => {{t,j ^ pi) and 
for all fc, i < A: < J => (t, k \= p)) 
there exists j,j < i ((^li H ‘4’) s'-nd 
for all fc, j < fc < i {t, k\= p)) 
for all p, {t, i \= {x 1 -^ p}P) 
there exists p, {t, i \= {x p}P) 


Fig. 7. Models relation for LTL used in the paper 


B Linear Temporal Logic 

Figure gives the definition of the models relaion for LTL as used in[^ Note 
that we assume an interpretation \= pi (B P 2 for the atomic propositions of the 
form Pi 0 p 2 - 


